The price of memory has dropped considerably in recent years. As a result, computers are being sold with ever greater amounts of memory. However, there remains a need for limiting, e.g., bounding, the amount of memory that may be used by a particular application, e.g., an image encoding application. By limiting the amount of memory required for an application, it makes it possible for the application to run in a predicable manner without concerns of running out of sufficient memory.
Image encoding can be a memory intensive process. The temporary buffering of image data, e.g., transform coefficients or other values used to represent an image, can involve vast amounts of data and thus require significant amounts of memory particularly if the image is large or is of very high resolution. Much of the image data which is buffered as part of an image encoding processes may be discarded and not included in the final set of image data output by the encoding processes, e.g., due to an image rate constraint which may be expressed as a total amount of memory, e.g., 10 MB, which is available to store the final set of encoded image data.
In many encoding applications it would be best if an image could be fully encoded and then data selected from the full set of encoded image data for inclusion in the final set of encoded image data. Such an approach allows for careful selection of the final encoded image data based on the knowledge about the encoded image data produced for the entire image. Unfortunately, in encoding embodiments where a complete image is processed and then a subset of the encoded data generated as part of the processing is selected to be included in the final encoded image data set, the amount of memory required for encoding grows in a manner which is normally proportionally to the image size. For large images the buffering requirements may exceed the amount of memory generally available in a personal computer and/or other system which may be used to perform the encoding.
Thus, as a practical matter, when encoding large or high resolution images which result in the generation of large amounts of data, it is often desirable from a memory management perspective to generate encoded image data corresponding to a portion of an image, store the generated encoded image data in a buffer, select some of the encoded image data to be included in the final set of encoded image data and discard the buffered encoded image data which has not been selected to be included in the final set of encoded image data. When such an approach is used, the amount of memory required for buffering image data as part of the encoding processes can be limited and need not grow in a manner which is proportional to the image size. This is because the full set of encoded image data generated as part of the process of encoding an image need not be stored at a single time. This allows large images to be encoded, e.g., a piece at a time, and/or images to be encoded in real time as portions of the image to be encoded, e.g., pixel values corresponding to a portion of the image, are received over a communications channel.
In order to control the amount of data stored, e.g., buffered, as part of an image encoding processes there is a need for buffer control methods and apparatus which can control the rate at which data is stored and removed from a buffer. While this might sound like a simple task, buffer rate control is complicated by the encoding objective of achieving a desired coding rate, e.g., bits per image portion, given a data constraint for the size of the encoded set of data representing the image while requiring image quality optimization. Mathematically, this is an optimization under constraints problem.
In view of the above discussion, it should be appreciated that there is a need for encoding methods and apparatus which incorporate rate control as well as buffering control features which allow for the encoding of large and/or high resolution images without concern that the size of the image being processed might cause a memory or buffer overflow.